The present invention generally relates to multi-dimensional graphics element displays, and more specifically, to displays of this type that are used to show multivariate time series data.
Rapid advances in electronic technology have led to a phenomena known as the information explosion. A major cause of this explosion is the increasing number of computer-aided sensing devices that gather information from the environment at astounding rates and in astounding amounts--rates and amounts too high for immediate assimilation by a human operator. Therefore, computers are often used to monitor the data that other computers are gathering in an effort to lower the total amount of information presented to an analyst or operator as a final product. With this pre-analyzed data, the analyst or operator can quickly make decisions about the environment.
Such information compression has been practiced for many years, but only recently has it received wide-spread recognition, often with the nomenclature of "Artificial Intelligence."
In an artificial intelligence system, a computer receives raw data and processes it using an algorithm that mimics human thought. Intuition and reasoning are reduced to programmable sets of logic and equations appropriate for the particular problem at hand, and this algorithm is then coded to operate as the raw data is acquired; that is, the data is processed in real time. When the sets of logic and equations are derived by observing the actions and reactions of a human expert (i.e., a radar expert, a meat packing expert), the artificial intelligence system is said to be an expert system. When the output of such a system is realized in a decision (i.e., yes, no, maybe), the system is said to be a decision-aid.
The premise of an artificial intelligence system is that the input data is too vast for humans to comprehend in raw form. Therefore, the input data cannot be displayed to a data analyst as is, but instead it must be pre-processed. This premise has been applied in many applications. For instance, a modern computer display of a battlefield situation gets very complicated very fast, as electronic intelligence sensors gather raw data at rates of thousands of reports per hour. The tactical decision-maker (the commander or intelligence officer) faces a complex, fast-changing display which often clouds the situation rather than clarifies it. Artificial intelligence eliminates selected information and presents only a distilled version to the commander. One consequence, of course, is that a computer algorithm now determines what is important and what is unimportant, and this is generally an undesirable situation. It would be better to have a human make such decisions, but showing all of the sensed data often produces an incomprehensible display.
The present invention challenges the basic idea that the human brain is incapable of processing high volumes of raw input data. Scientists today do not fully understand the capabilities of the human brain, but most agree that it possesses more capability than what is normally used in everyday thought. In particular, many experts believe that the human brain is much better at recognizing patterns than are even the best of contemporary computers.
For example, if an expert system is presented with unforeseen circumstances, it normally either conclude that there are no rules applicable and prompts the human user for new rules, or it adopts existing rules, which might result in a false interpretation of the tactical situation. Either course of action would cause a delay or incorrect response to a critical problem
If the data is presented to the human operator in an intelligent and comprehensive way, often even highly unusual situations may be easily recognized by the human brain. Instead of concentrating on building more and more elaborate systems of rules, there must be an effort to accommodate the innate and vast human perceptual capability. The deficiency in many computer graphics presentations is not in the output volume, but in the display itself. More intelligent computer programs are not needed, but more intelligently-designed computer displays are.
Dimensionality is always the first problem met while designing any kind of graphical representation of data. If there are two parameters or dimensions, say x and y, the solution is easy. But the present invention addresses data with k parameters or dimensions, k being greater than two. For example, in an ELINT (Electronic Intelligence) sensor system monitoring a ground battlefield situation, computerized data reports may include the following dimensions: (1) the radio frequency of a detected radar signal, (2) the pulse interval of that signal, (3) the pulse duration, (4) the received signal, (5) two or three position dimensions of the signal source, (6) a velocity dimension if the signal source is moving, and (7) instantaneous performance parameters of the sensing device. To fully utilize such a piece of raw data, all of the sensed dimensions should be available for inspection simultaneously, although some of the dimensions are more important than others.
Several creative prior art approaches have been suggested for simultaneously displaying k dimensions of data. With one approach, disclosed in "Pattern and Process of the Evolution of Human Septic Shock," by Siegel, et al., Surgery, Volume 70, Number 2, August 1971, page 232, a circle is drawn and then points are marked along k equally space rays from the center of the circle. The distance from the circumference to each point is equal to standardized distances from the means of the k variables. The points are connected to form a polygon, and the resulting polygons assume meaningful shapes, easily recognized and grouped by the human thought processes. A more standard approach is the use of profiles. With this approach, one represents a point in k-dimensional space by a series of k bars of heights corresponding to the values of the variables, standardized in some way, and often plotted symmetrically about the mean. Yet another method disclosed in "Plots of High Dimensional Data" by D. F. Andrews, Biometrics, Volume 28, March 1972, page 125, uses plots of a Fourier series which produces curves suggestive of the characteristics of the data.
One of the most effective, as well as the most humorous, representation of data is by faces according to a procedure discussed in "The Uses of Faces to Represent Points in a k-Dimensional Space Graphically," by H. Chernoff, Journal of the American Statistical Association, Volume 68, Number 342, June 1973, page 361. With this procedure, a computer is used to draw a small cartoon face whose features, such as the length of the nose and the curvature of the mouth, correspond to the components of a data point in k-space. The human brain is so adept at processing facial data that pattern recognition is quite easy. Happy data points are quickly discerned from sad ones and sly ones. Groupings in the data form rapidly. This technique is basically the opposite of a common artificial intelligence technique. Rather than employ a computer to distinguish between human faces by some type of algorithm or arithmetical computation, Chernoff's procedure uses a computer to draw a face to help an operator analyze raw data.
Another approach to displaying multiply-dimensioned data utilizes the Lissajous patterns, first studied by J. A. Lissajous during the mid-1800s. Plotting two sine waves against each other: EQU x(t)=Ax sin (Bxt+Cx)+Dx EQU y(t)=Ay sin (Byt+Cy)+Dy,
t in the interval (-E,E)
produces an interesting pattern with information about the nine A, B, C, D, and E parameters, placing particular emphasis on the ratio of Bx to By. Additional information can be incorporated into such Lissajous patterns by parametrically varying the form of the plotted function from, for example, a sine wave to a triangular wave to a square wave. Cardioids and similar figures also have characteristic shapes which may prove useful in this problem.
The discussion above presents a very simplified approach to multiple dimensional data representations. It is limited to two dimensions. It is believed that holography will alleviate the two-dimensional constraint eventually, but color dimensions can be used in the meantime to represent dimensions greater than two. A full-color, multiple dimension display coupled with flexibility and power from a modern high resolution graphics device, provides a potentially viable method for transferring information from raw data sources to the human analyst very efficiently.
A procedure for representing multivariate time series data by means of interactive, computer-generated dynamic imagery with computer music accompaniment is disclosed in "Dynamic Representation of Multivariate Time Series Data," by Mezrich, et al., Journal of the American Statistical Association, Volume 79, No. 385, March 1984, Page 34. Their display depicts the data as dancing, multi-colored line segments arranged to give a sense of depth associated with the instantaneous value of the variables. The data analyst is given a data table for interacting with the system and may, among other things, scan through the data in a forward or reverse direction. Mezrich has experimentally demonstrated the superiority of such dynamic representations over traditional static ones when discerning positive, pairwise data correlations in time series.
A practical extension to the display forms discussed above is to show a three-dimensional figure in two dimensions. For instance, three-dimensional cartoon faces, and three-dimensional Lissajous patterns are obvious extensions of the above-discussed display procedures. Currently available computer systems may be used to transform a three-dimensional figure to a two-dimensional figure, such as by orthographic or perspective projections; and moreover, the two dimension figure can be easily manipulated to simulate rotation, translation and magnification or reduction of the three-dimensional figure.
Color may be used to add additional dimensions to a data display; and, for example, the intensity, hue and saturation of a color are each clearly perceived by a human and may be used to represent additional, independent dimensions. M. D. Buchanan, in "Effective Utilization of Color in Multidimensional Data Presentations," SPIE Vol. 199 Advances in Display Technology (1979), Page 9, reports that the standard red, green, and blue dimensions (rgb) are not easily perceived independently by a human, but intensity, hue, and saturation (ihs) are.